Near-Eye Display Gaze Tracking via Convolutional Neural Networks

Virtual reality and augmented reality systems are currently entering the market and attempt to mimic as many naturally occurring stimuli in order to give a sense of immersion. Many aspects such as orientation and positional tracking have already been reliably implemented, but an important missing piece is eye tracking. VR and AR systems equipped with eye tracking could provide a more natural interface with the virtual environment, as well as opening the possibility for foveated rendering and gaze-contingent focus. In this work, we approach an eye tracking solution specifically designed for near-eye displays via convolutional neural networks, that is robust against lighting changes and occlusions that might be introduced when placing a camera inside of a near-eye display. We create a new dense eye tracking dataset specifically intended for neural networks to train on. We present the dataset as well as report initial results using this method.